The Inverse G‐Wishart distribution and variational message passing

Variational Message Passing

Bayesian inference is now widely established as one of the principal foundations for machine learning. In practice, exact inference is rarely possible, and so a variety of approximation techniques have been developed, one of the most widely used being a deterministic framework called variational inference. In this paper we introduce Variational Message Passing (VMP), a general purpose algorithm...

d-VMP: Distributed Variational Message Passing

Motivated by a real-world financial dataset, we propose a distributed variational message passing scheme for learning conjugate exponential models. We show that the method can be seen as a projected natural gradient ascent algorithm, and it therefore has good convergence properties. This is supported experimentally, where we show that the approach is robust wrt. common problems like imbalanced ...

Non-conjugate Variational Message Passing for Multinomial and Binary Regression

Variational Message Passing (VMP) is an algorithmic implementation of the Variational Bayes (VB) method which applies only in the special case of conjugate exponential family models. We propose an extension to VMP, which we refer to as Non-conjugate Variational Message Passing (NCVMP) which aims to alleviate this restriction while maintaining modularity, allowing choice in how expectations are ...

Stein Variational Message Passing for Continuous Graphical Models

We propose a novel distributed inference algorithm for continuous graphical models, by extending Stein variational gradient descent (SVGD) (Liu & Wang, 2016) to leverage the Markov dependency structure of the distribution of interest. Our approach combines SVGD with a set of structured local kernel functions defined on the Markov blanket of each node, which alleviates the curse of high dimensio...

Centralized Estimation of Adhesion Loss in Wheel – Rail System Using Variational Bayes and Variational Message Passing